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(57) A signal adaptive loop filter reduces blocking 
effects and ringing noise of motion-compensated image 
data. In the loop filter, binary edge map information on 
which global thresholding and local thresholding with 
respect to the motion-compensated image are 
reflected, is used to judge whether the binary edge map 
information within a predetermined size of the filter win- 
dow is either a homogeneous or an edge area. If it is 
judged as a homogeneous area, predetermined first 
weight factors corresponding thereto are used for gen- 



erating a filtered pixel value corresponding to the corre- 
sponding filter window. While, in the case that it is 
judged as an edge area, predetermined second weight 
factors are varied according to the binary edge map 
information corresponding to pixels, and the filter win- 
dow having the varied second weight factors is used for 
generating the filtered pixel value corresponding to 
respective corresponding pixels. 



FIG. 2 



MOTION 
COMPENSATED — 
IMAGE DATA 



310 

\ 

r 

TO SUBTRACTER! 



IMAGE 
STORE 



20 



GRADIENT 
IMAGE 
GENERATOR 



GLOBAL 
H EDGE 
MAPPER 



100 
30 I 



fji 



40 



200 
60 



OR 



LOCAL 
H EDGE 
MAPPER 



-H- 

J 

l 

7ZT- 



FILTER 
DETERMINER 



70 



AVERAGE 
FILTER 



80 



WEIGHTED 
FILTER 



Printed by Xerox (UK) Business Services 
2.15.7/3.4 



BNSDOCtD: <EP 0817497A2J__> 



1 



EP 0 817 497 A2 



2 



Description 

The present invention relates to a loop filtering 
method capable of reducing blocking effects and ringing 
noise, and more particularly, to a loop filtering method 5 
capable of reducing blocking effects and ringing noise of 
a motion-compensated image. 

Most of picture coding standards including H.263 of 
International Telecommunication Union (ITU) and 
MPEG-1 and MPEG-2 of Organization for Standardiza- 10 
tion (ISO), use the block-based processing for motion 
estimation and discrete cosine transform (DCT). 

This block-based processing induces the well- 
known blocking effects and ringing noise, in particular, 
when an image is highly compressed. The blocking is 
effects induce a viewer who views a decompressed 
image displayed on a screen, to feel the boundary 
between the blocks in the visual sense. Typical blocking 
effects are grid noise in the monotone area and stair- 
case noise along the image edges. The ringing noise 20 
induces sinusoidal traces on the practical edges of an 
image. 

To solve the above problem, it is an object of the 
present invention to provide a loop filtering method 
capable of largely reducing blocking effects and ringing 25 
noise, by adaptively filtering a motion-compensated 
image data stored in a video memory of encoding and 
decoding systems. 

To accomplish the above object of the present 
invention, there is provided a signal adaptive loop filter- 30 
ing method of largely reducing blocking effects and ring- 
ing noise, comprising the steps of: 

(a) generating binary edge map information by glo- 
bally thresholding and locally thresholding the 35 
motion-compressed image; 

(b) judging whether the binary edge map informa- 
tion within a filter window corresponding to respec- 
tive pixels is either homogeneous area or edge 
area, by using the binary edge map information 40 
belonging to a predetermined size of the filter win- 
dow; 

(c) generating a filtered pixel value corresponding 
to a respective pixel using the filter window which 
has predetermined first weight factors related to the 45 
homogeneous area, when the binary edge map 
information corresponding to the respective pixel is 
judged as the homogeneous area in said step (b); 

(d) altering predetermined second weight factors 
according to the binary edge map information cor- so 
responding to the respective pixel when the binary 
edge map information is judged as the edge area in 
said step (b); and 

(e) generating a filtered pixel value corresponding 
to the respective pixel, using thef ilter window which 
has the altered second weight factors in said step 
(d). 



By way of example, a specific embodiment of the 
invention will now be described, with reference to the 
accompanying drawings wherein : 

Figure 1 is a block diagram showing an encoding 
apparatus employing a loop filter according to the 
preferred embodiment of the present invention; 

Figure 2 is a block diagram illustrating a loop filter 
according to the preferred embodiment of the 
present invention; 

Figure 3 is a conceptual view showing a binary 
edge map produced in a binary edge mapper of 
Figure 2; 

Figure 4A shows a filter window for a 3x3 filter pro- 
posed in the present invention; Figure 4B shows 
weight factors for a 3x3 average filter; and Figure 
3C shows weight factors for a 3x3 weighted filter. 

Figure 1 is a block diagram showing an encoding 
apparatus employing a loop filter according to a pre- 
ferred embodiment of the present invention, which 
shows an encoding apparatus related to H.263. In the 
Figure 1 apparatus, since blocks excepting a loop filter 
310 proposed by the present invention are well-known 
to those who are concerned with high efficiency encod- 
ing systems including a prediction encoder, the opera- 
tion with regard to such blocks will be described briefly. 

Referring to Figure 1 , an input video signal is input 
to a subtracter 301, a first switch 302 and a prediction 
encoder 309. The subtracter 301 subtracts a filtered sig- 
nal by the loop filter 310 from the input video signal, an 
output signal of which is applied to the first switch 302. 
The first switch 302 selects either the input video signal 
or the output signal of the subtracter 301 according to 
the control of an encoding controller 305, and the 
selected signal is applied to an orthogonal transformer 
303. The encoding controller 305 controls the first 
switch 302 so that the input video signal is applied to the 
orthogonal transformer 303 in the case of an intra cod- 
ing mode, and the output signal of the subtracter 301 is 
applied to the orthogonal transformer 303 in the case of 
an inter coding mode. A signal passing through the 
orthogonal transformer 303 and a quantizer 304 is 
applied to a video multiplexer encoder (not shown) and 
a dequantizer 306. The dequantizer 306 dequantizes 
the applied signal, and the dequantized signal is sup- 
plied to an inverse-orthogonal transformer 307. An 
adder 308 adds an output signal of a second switch 311 
and that of the inverse-orthogonal transformer 307. and 
the added signal is supplied to the prediction encoder 
309. The prediction encoder 309 performs a motion 
55 estimation and compensation using the input video sig- 
nal and a video signal stored in an internal memory, and 
the resultant signal by the motion estimation and com- 
pensation is stored in the internal memory. The predic- 



2 



BNSQOCID: <EP 0817497A2_I_> 



3 



EP 0 817 497 A2 



4 



tion encoder 309 also outputs motion vector V obtained 
by the motion estimation to the video multiplexer 
encoder. The loop filter 310 in accordance to the 
present invention described below performs the filtering 
with regard to the picture stored in the prediction 5 
encoder 309. An output signal of the loop filter 310 is 
supplied to the subtracter 301 and the second switch 
31 1 . The second switch 31 1 which is controlled by the 
encoding controller 305, cuts off what the input signal 
has supplied to the adder 308 in the case of the intra 70- 
coding mode, and makes the input signal supplied to 
the adder 308 in the case of the inter coding mode. The 
encoding controller 305 controls the above operation of 
Figure 1 apparatus, generates a flag P representing 
intra/inter modes, a quantizer indicator signal qz and a 15 
flag t representing whether or not a signal is transmit- 
ted. 

Figure 2 is a block diagram showing the loop filter 
310 of Figure 1 apparatus. The loop f ilter 310 of Figure 
2 includes an image store 10, a binary edge mapping 20 
unit 100 and a signal adaptive filter portion 200. The 
image store 10 stores data supplied from the prediction 
encoder 309 in which the motion estimation and com- 
pensation encoding are performed. The binary edge 
mapping unit 1 00 includes a gradient image generator 25 
20, a global edge mapper 30. a local edge mapper 40 
and a logical OR unit 50, which generates binary edge 
map information on which global edges and local edges 
of a motion-compensated image are reflected. The sig- 
nal adaptive filter portion 200 includes a filter deter- 30 
miner 60, an average fflter 70 and a weighted filter 80, 
which filters the motion -compensated image data using 
one of the average filter and the weighted filter to be 
described below based on the binary edge map infor- 
mation. The grid noise and the ringing noise are miti- 35 
gated in the signal adaptive filter portion 200. 

The gradient image generator 20 generates gradi- 
ent image data using the image data stored in the image 
store 10. The gradient image generator 20 applies a 
vertical Sobel gradient operator and a horizontal Sobel 40 
gradient operator to the motion-compensated image, 
that is, the image data stored in the image store 10, so 
as to generate a gradient image. The gradient image 
data obtained by the gradient image generator 20 is 
supplied to the global edge mapper 30 and the local 45 
edge mapper 40. 

The global edge mapper 30 produces a global edge 
map with respect to the whole image, and the local edge 
mapper 40 divides the whole image into blocks having a 
predetermined magnitude and produces a local edge so 
map with respect to each of blocks. In more detail, the 
global edge mapper 20 calculates an edge value (i, j) 
corresponding to each pixel using the following equation 

0). 

55 

If |v h | + |v v |^T g edge(i,j) = 1 (1) 
Otherwise edge (i, j) = 0 



Here v h and v v represent a horizontal gradient 
image and a vertical gradient image at a location (i, j) 
obtained by the horizontal Sobel gradient operator and 
the vertical Sobel gradient operator, respectively. | v h | + 
| v v | represents a gradient value at a location (i, j) and 
T g is a global threshold value which is "100" when each 
pixel within an image has 256 gray levels. 

Therefore, if a gradient value corresponding to a 
pixel of the location (i, j) is larger than or equal to the 
global threshold value T g , the global edge mapper 30 
determines an edge value corresponding to the pixel as 
"1 ". On the other hand, if a gradient value corresponding 
to a pixel is smaller than the global threshold value T g , 
the global edge mapper 30 determines an edge value (i, 
j) corresponding to the pixel as "0". The global edge 
map information obtained by applying the above equa- 
tion to the whole image is supplied to the logical OR unit 
50. 

The local edge mapper 40 produces a local edge 
map using the output of the gradient image generator 
20. The local edge mapper 40 calculates a local thresh- 
old value with respect to each of all M-j x M 2 blocks of 
the gradient image, and calculates local edge values 
with respect to all gradient values contained in corre- 
sponding block using the calculated local threshold 
value. The block-based processing techniques, such as 
DCT transform and quantization, process signals with 
respect to an 8x8 block containing basically 8x8 pixels. 
Thus, the local edge mapper 40 according to the one 
embodiment of the present invention is also designed to 
draw a local edge map by using an 8x8 block. However, 
it will be apparent to a person skilled in the art that the 
present invention is not limited to the block of such a 
size. 

The local threshold value T n with respect to a n-th 
8x8 block in the gradient image is defined by the follow- 
ing equations (2) - (4). 



wherein, 




™n = J I 0(U> W 

0.j)€ R n 

Here, g (i.j) represents a gradient value, R n repre- 
sents a n-th 8x8 block region m n and o n represent a 
mean value and a standard deviation of a n-th 8x8 block 
of the gradient image, respectively. T g represents a glo- 
bal threshold value and N is 64 in the case of an 8x8 
block. If the n-th' 8x8 block is homogenous, a ratio a n /m n 
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tends to be "0" with the result that T n becomes very 
close to T g . Meanwhile, if the rvth 8x8 block is part of a 
complicated image, the ratio o^/m,, increases. As a 
result. T n becomes smaller than T g . This small T n is 
used in drawing a detail edge map which is not classi- 
fied as a global edge by T g . 

The local edge mapper 40 individually compares 
the local threshold value T n with respect to the n-th 8x8 
block with a part of the gradient values within the block, 
in magnitude. Here, the part of the gradient values cor- 
responds to 6x6 pixels within the 8x8 block excluding 
the boundary pixels within the 8x8 block. If the gradient 
value used for drawing the local edge map is defined as 
described above, the detailed information is protected 
from blurring, and the grid noise is prevented from being 
detected as an image edge. If each gradient value being 
allowable within the n-th 8x8 block region R n is larger 
than or equal to the local threshold value T n , the local 
edge mapper 40 determines a local edge value corre- 
sponding to the gradient value as "1". Meanwhile, if a 
gradient value corresponding to a pixel is smaller than 
the local threshold value T n , the local edge mapper 40 
determines a local edge value corresponding to the gra- 
dient value as "0". The local edge mapper 40 generates 
local edge map information by performing the operation 
procedure including an 8x8 block division, local edge 
value calculation and calculation of a local edge value 
using allowable gradient values of each block, with 
respect to all gradient values generated by the gradient 
image generator 20. The generated local edge map 
information is supplied to the logical OR unit 50. 

The logical OR unit 50 performs a logical OR oper- 
ation of the global edge map information and the local 
edge map information which are related to each other 
with respect to a pixel location. The logical OR unit 50 
performs a logical OR operation with respect to all glo- 
bal edge values on the global edge map and all local 
edge values on the local edge map, and outputs binary 
edge map information representing the result to the filter 
detei miner 60. The binary edge map produced by the 
logical OR unit 50 is conceptually shown in Figure 3. On 
the binary edge map of Figure 3, the shaded squares 
are represented the edge points each of which has the 
edge value "1". 

The filter determiner 60 stores the binary edge map 
information supplied from the logical OR unit 50. The 
motion compensated image is classified into two areas 
such as an edge area and a homogeneous area by a 
binary edge map. For this classification, the present 
invention uses the filter determiner 60. The average fil- 
ter 70 and the weighted filter 80 described later use a 
3x3 filter window. Thus, a filter window used in the filter 
determiner 60 has a 3x3 size, too. 

The filter determiner 60 judges whether a region on 
the binary edge map where the filter window is located 
is an edge area or a homogeneous area, based on the 
edge values contained in the filter window having a pre- 
determined size, that is the 3x3 size. If it is judged as a 



6 

homogeneous area, the filter determiner 60 outputs 
position data with respect to the central point in the filter 
window used for judgement, to the average filter 70. In 
the case that it is judged as an edge area, the filter 

5 determiner 60 outputs the binary edge map information 
and position data with respect to the central point in the 
filter window used for judgement, to the weighted filter 
80. Here, the centra! point represents a point where the 
pixel value located at the point is replaced with a new 

w value by filtering. 

Figures 4A-4C relate to a two-dimensional 3x3 fil- 
ter. Figure 4A shows a filter window for a 3x3 filter. Fig- 
ure 4B shows the weight factors for the 3x3 average 
filter, and Figure 4C shows the weight factors for the 3x3 
is weighted filter. It is allowable that the weight factor of the 
central point shown in Figure 4C is replaced by 2. In the 
filter window of Figure 4A, the central point of the filter 
window is a point where an index value of the filter 
weight factor is "2". 
20 The average filter 70 and the weighted filter 80 are 
a sort of two-dimensional low-pass filters, the operation 
of which will be described as follows. 

If position data with respect to the central point is 
input, the average filter 70 reads the pixel values neces- 
25 sary to calculate the filtered pixel value of the central 
point from the image store 10. Then, the average filter 
70 calculates the filtered pixel values using the read 
pixel values and the weight factors shown in Figure 4B. 
The calculated filtered pixel value is used as an altered 
30 pixel value with respect to the central point. The 
weighted filter 80 performs a filtering operation based 
on the binary edge map information and the position 
data supplied from the filter determiner 60 with respect 
to the central point. The operation of the weighted filter 
80 will be described below in more detail. 

If the central point "5" of the filter window is an edge 
point, the weighted filter 80 does not perform a filtering 
operation for the central point. If the edge point or the 
edge points are positioned in the 3x3 filter window 
excluding the central point, the weighted filter 80 per- 
forms a filtering operation using the weight factors 
shown in Figure 4C. 

If any edges are on the points 2 and 6, 6 and 8, 4 
and 8 or 2 and 4 in Figure 4A, the weight factors of the 
edge pixel and the outer neighbour pixels are set to 
zero. The signal adaptive filtered image data obtained 
by the average filter 70 and the weighted filter 80 are 
supplied to the image store 10. 

The image store 10 replaces a corresponding pixel 
value by using the image data supplied from the aver- 
age filter 70 and the weighted filter 80. Accordingly, the 
image store 10 stores the signal adaptive filtered image 
data with respect to all pixels in the image applied from 
the prediction encoder 309. The image store 10 outputs 
image data to be used in the subtracter 301 and the 
adder 308 of Figure 1 . 

Although the above one embodiment illustrates the 
loop filtering with regard to an image output from a pre- 
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diction encoder, it will be obvious to a person skilled in 
the art that a modification of performing loop filtering 
with respect to the image stored in a frame memory of 
the prediction encoder is made. 

Although the above embodiment is illustrated with 
reference to an encoder, it will be apparent to a person 
skilled in the art that the loop filter can be also used in 
motion-compensated image data of a decoding appara- 
tus. Furthermore, description relating to H.263 in this 
embodiment is made, but verification model VM 2. 1 of 
MPEG-4 can be equally used. 

As described, the loop filtering method according to 
the present invention reduces the blocking effects and 
the ringing noise from the motion -compensated image, 
thereby providing an effect of improving the quality of 
the decompressed image. 

While only certain embodiments of the invention 
have been specifically described herein, it will be appar- 
ent that numerous modifications may be made thereto 
without departing from the spirit and scope of the inven- 
tion. 

The reader's attention is directed to all papers and 
documents which are filed concurrently with or previous 
to this specification in connection with this application 
and which are open to public inspection with this speci- 
fication, and the contents of all such papers and docu- 
ments are incorporated herein by reference. 

All of the features disclosed in this specification 
(including any accompanying claims, abstract and 
drawings), and/or all of the steps of any method or proc- 
ess so disclosed, may be combined in any combination, 
except combinations where at least some of such fea- 
tures and/or steps are mutually exclusive. 

Each feature disclosed in this specification (includ- 
ing any accompanying claims, abstract and drawings), 
may be replaced by alternative features serving the 
same, equivalent or similar purpose, unless expressly 
stated otherwise. Thus, unless expressly stated other- 
wise, each feature disclosed is one example only of a 
generic series of equivalent or similar features. 

The invention is not restricted to the details of the 
foregoing embodiment(s). The invention extends to any 
novel one, or any novel combination, of the features dis- 
closed in this specification (including any accompanying 
claims, abstract and drawings), or to any novel one, or 
any novel combination, of the steps of any method or 
process so disclosed. 

Claims 

1 . A signal adaptive loop filtering method for a motion- 
compensated image, said signal adaptive loop fil- 
tering method comprising the steps of: 

(a) generating binary edge map information by 
globally thresholding and locally thresholding 
the motion-compressed image; 

(b) judging whether the binary edge map infor- 



mation within a filter window corresponding to 
respective pixels is either homogeneous area 
or edge area, by using the binary edge map 
information belonging to a predetermined size 
5 of the filter window; 

(c) generating a filtered pixel value correspond- 
ing to a respective pixel using the filter window 
which has predetermined first weight factors 
related to the homogeneous area, when the 

10 binary edge map information corresponding to 

the respective pixel is judged as the homoge- 
neous area in said step (b); 

(d) altering predetermined second weight fac- 
tors according to the binary edge map informa- 

75 tion corresponding to the respective pixel when 

the binary edge map information is judged as 
the edge area in said step (b); and 

(e) generating a filtered pixel value correspond- 
ing to the respective pixel, using the filter win- 

20 dow which has the altered second weight 

factors in said step (d). 

2. The signal adaptive filtering method according to 
claim 1 , wherein said step (a) comprises the steps 
25 of : 

(a1) generating a gradient image correspond- 
ing to the motion-compressed image; 
(a2) generating global edge map information 

30 which is composed of edge values correspond- 

ing to the respective pixels of the motion-com- 
pressed image, by comparing the 
predetermined first thresholding value with a 
gradient value corresponding to the respective 

35 pixels within the gradient image; 

(a3) generating local edge map information 
which is composed of edge values correspond- 
ing to the respective pixels of the motion-com- 
pressed image, by comparing gradient values 

40 within a predetermined size of individual blocks 

into which the motion-compressed image is 
divided, with a predetermined second thresh- 
olding value corresponding to the block; and 
(a4) logically summing an edge value in the 

45 global edge map information, and a corre- 

sponding edge value in the local edge map 
information, and generating binary edge map 
information. 

so 3. The signal adaptive filtering method according to 
claim 2, wherein said step (a3) calculates the sec- 
ond thresholding value corresponding to respective 
blocks using the following equation : 

55 <j n 

wherein, 
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m n = ^j £ g(U) 

G,j)eR n 

Here. g(U) represents a gradient value, R n 
represents a n-th 8x8 block region, m n and a n rep- 
resent a mean value and a standard deviation of a 
n-th 8x8 block of the gradient image and T g repre- 
sents a global threshold value. 

4. The signal adaptive filtering method according to 
claim 3, wherein said step (a3) compares the gradi- 
ent values corresponding to 6x6 pixels within a 8x8 
block excluding boundary pixels within the 8x8 
block with the corresponding second thresholding 
value. 

5. The signal adaptive filtering method according to 
claim 1, wherein the more the predetermined sec- 
ond weight factors move from the centre of the filter 
window to the boundary thereof, the less the prede- 
termined second weight factors have. 

6. The signal adaptive filtering method according to 
claim 5, wherein said filter window has a 3x3 size. 

7. The signal adaptive filtering method according to 
claim 6. wherein said step (c) uses a filter window 
having the same magnitude of first weight factors. 

8. The signal adaptive filtering method according to 
claim 7, wherein the magnitude of said first weight 
factors is "1". 

9- The signal adaptive filtering method according to 
claim 8, wherein a second weight factor corre- 
sponding to said central point is "2" in magnitude, 
and second weight factors corresponding to the 
others thereof are" 1". 

10. The signal adaptive filtering method according to 
claim 8, wherein a second weight factor corre- 
sponding to said central point is "3" in magnitude, 
and second weight factors corresponding to the 
others thereof are "1 

11. The signal adaptive filtering method according to 
claim 8, wherein said step (d) comprises the step of 
resetting, as zero, second weight factors corre- 
sponding to positions put on a diagonal line and 
second weight factors corresponding to the outer 
neighbour of positions placed on the diagonal line, 
if the binary edge map information which corre- 



sponds to positions on the diagonal lines among 
the positions excluding the central point of the 3x3 
sized filter window is represented as all edge 
points. 
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